Evolutionary Design of Numerical Methods: Generating Finite Difference and Integration Schemes by Differential Evolution

Classical and new numerical schemes are generated using evolutionary computing. Differential Evolution is used to find the coefficients of finite difference approximations of function derivatives, and of single and multi‐ step integration methods. The coefficients are reverse engineered based on samples from a target function and its derivative used for training. The Runge‐Kutta schemes are trained using the order condition equations. An appealing feature of the evolutionary method is the low number of model parameters. The population size, termination criterion and number of training points are determined in a sensitivity analysis. Computational results show good agreement between evolved and analytical coefficients. In particular, a new fifth‐order Runge‐Kutta scheme is computed which adheres to the order conditions with a sum of absolute errors of order 10 ‐14. Execution of the evolved schemes proved the intended orders of accuracy. The outcome of this study is valuable for future developments in the design of complex numerical methods that are out of reach by conventional means. Introduction In this paper coefficients of classical numerical schemes for differentiation and integration are generated by an evolutionary algorithm (EA). The widespread use and therefore importance of finite difference methods and Runge‐Kutta methods is self‐evident. Basic finite difference schemes have straightforward analytical derivations, but complex computational science problems often require the design of more sophisticated methods lacking standard approaches. The novel perspective investigated in this study embraces the application of EAs for designing numerical formulae. Runge‐Kutta methods comprise a family of integration schemes for solving initial value problems of Ordinary Differential Equations (ODEs), the most famous being the fourth‐order RK4 Runge‐Kutta method. For orders past two the systems of order condition equations are underdetermined, this hampers analytical derivation. Tsitouras & Famelis (2012) note that traditional heuristics based on computing Jacobians are less suitable for finding Runge‐Kutta schemes than EAs, since determining the derivatives of the variables as they appear in the order conditions is cumbersome (Rothlauf 2011). In the last couple of decades several classes of EAs have been developed and applied to an abundance of fields in science and engineering (Eiben & Smith 2013). The power of evolutionary computing is increasingly reflected in its ability to tackle more general computational problems that are analytically intractable and, moreover, inaccessible to conventional numerical methods. In a seminal double pendulum experiment Schmidt & Lipson (2009) showed the effectiveness and versatility of symbolic regression based on Genetic Programming (GP) by deriving …